@Dynamic Sparse Training

Researchers at arXiv have identified that Batch Normalization causes gradient skew in dynamic sparse training (DST) methods, leading to slower convergence compared to dense neural network training. Th…